The present disclosure relates to an image processing device to execute image processing with use of plural parallax images, an image processing method, an image processing program, and an imaging device including such an image processing device.
In the related art, algorithms of distance calculation by various kinds of image processing have been proposed and developed. As one among them, a distance calculation method employing a technique called block matching (stereo matching) has been proposed (refer to e.g. D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” IJCV 2002).
In this technique, the distance is calculated by image processing with use of plural parallax images having parallaxes different along a certain direction (e.g. horizontal direction). Specifically, the amount of movement of an object (phase difference between parallax images) is obtained by sequentially comparing the parallax images in a local area (obtaining a correlation value), and the distance is calculated based on this phase difference.
In such a stereo matching technique, in the case of calculating the distance from plural parallax images along the horizontal direction (H-direction) in the image, the local area (unit area) as the comparison subject is sequentially moved in the horizontal direction in the obtaining of the phase difference between parallax images. Furthermore, the positional displacement (pixel displacement) between the parallax images regarding the unit areas having the highest correlation in the comparison range is obtained as the phase difference. In the case of calculating the distance from plural parallax images along the vertical direction (V-direction) in the image, similarly the unit area as the comparison subject is sequentially moved in the vertical direction in the obtaining of the phase difference between parallax images.
However, such a related-art stereo matching technique has the following problem. Specifically, for example if an edge area is included in the image, the correct phase difference can not be obtained in this edge area, and as a result, it is difficult to achieve the correct phase difference distribution in the image. If the correctness of the phase difference distribution is low, distance information (distance distribution in the image) obtained based on the phase difference distribution is also incorrect.
This problem is attributed to the following reason. Specifically, for example if an edge area along the horizontal direction (horizontal edge area) exists, the correlation is always high (correlation value is always constant in the comparison range) although the local area as the comparison subject is moved in this edge area (moved in the horizontal direction). That is, the positional displacement between the unit areas having the highest correlation can not be uniquely decided, so that the phase difference can not be defined.
There is a need to provide an image processing device capable of generating more correct phase difference distribution compared with the related art, an image processing method, an image processing program, and an imaging device including such an image processing device.